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(54) A system for accessing multimedia mailboxes and messages over the internet and via 
telephone 



(57) A unified messaging system that provides a 
multimedia mailbox. The system allows a subscriber to 
access scored multimedia messages, such as voicemail 
messages, facsimile messages, combined voice and 
facsimile messages and video messages, not only 
through a public switched telephone network using a tel- 
ephone but also over a data network, such as the Inter- 
net or an intranet, using a personal computer. The sys- 
tem provides voicemail access over the telephone net- 
work, indicating message number, etc. with the ability to 
play messages to the telephone user as desired. For 
text type messages, such as facsimile and e-mail, the 
system converts the text into speech and plays the 



speech to the telephone user The system allows a per- 
sonal computer user to obtain the data network access 
using an Internet browser. The browser is used to ac- 
cess a home page of the system and get information 
about the messages stored, and is used to download 
(get) and play the messages the personal computer 
via data streaming in the case of a voice or video mes- 
sages or view the messages in the case of text type 
messages, such as facsfrnile and e-mail. The user can 
also perform the other typical messaging functions over 
the data network connection that are provided for tele- 
phone access, such as viewing a message list, saving 
and deleting messages, group list admlriistratkxi and 
other administration tasks. 
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De cription 

REFERENCE TO MICROFICHE APPENDIX 

A microfiche appendix having 6 nnicrofiche and 490 
frames is included herewith that includes source code 
in the C^VHTML languages. 

BACKGROUND OF THE INVENTION 

Field of the invention 

The present invention is directed to a system for ac- 
cessing stored messages over a network and. more par- 
ticularly, is directed to a system for providing unified ac- 
cess to stored messages, such as multimedia mail mes- 
sages, in a unified multimedia mailbox through multiple 
access pathways such as over a telephone network us- 
ing a telephone and over the Internet using a browser, 

Description of the Related Art 

Communication systems currently exist that allow 
different types of messages, such as voice mail mes- 
sages and facsimile messages, to be stored for later re- 
trieval by a subscriber to such systems. These types of 
systems are described in U.S. patents 5,029.1 9g; 
5.193.110; 5.260,990: 5.263,080; 5.475.748; 
5,493.607; 5.524.139; 5,519 766 and 5,008.926. alt in- 
corporated by reference herein These systems allow a 
caller or sender to leave a message, such as a voice 
mail message, for a subscriber whenever the subscriber 
is not available. When a voice mail message is to be 
retrieved the subscriber typically connects with the sys- 
tem over a conventional telephone line via a telephone 
call and plays the message by using the touchtones pro- 
duced by the telephone to control playback, as well as 
other functions. In these systems ihe access by the sub- 
scriber is typically only through a telephone line connec- 
tion. Today, there is a need to allow access to such sys- 
tems through other means such as the Inlemet or In- 
tranet. 

Several different types of messaging systems, such 
as voice mail and e-mail are also available to users. 
Users of the variety of today's messaging systems typ- 
ically have to use several different systems and/or ter- 
minals to get their messages. A typical business user 
may have several voice mailboxes, several e-mail mail- 
boxes, and perhaps some mailbox-like facsimile servic- 
s. Each of these mailboxes requires separate opera- 
tions and different types ol terminals (DTMF telephone 
for voice mail personal computer (PC) for e-mail ac- 
cess, facsimile machine/PC for facsimile messages). 
Th mailboxes hav diff rent names (addr ss s) and 
cannot usually inlerwork. Notification m chanisms are 
either non-existent, or tied to on of the mailboxes. What 
is needed is a mailbox system that int grates all of th se 
message types and access methods. 
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SUMf^ARY OF THE INVENTION 

It is an object of the present invention to provide a 
system that allows e subscriber to access stored mes- 
5 sages over not only a telephone network but also over 
a network such as the Internet or an intranet. 

It is an additional object of the present inVerition to 
provide a system that unifies messeige storage allowing 
different typ§s of messages or electronic communica- 
10 tions such as voicemaiL facsimile, e-mail and video mai I 
to be stored on a single system in a single unified mul- 
timedia mailbox, and accessed via different pathwaySi 
such as via a telephone network or the Internet/Intranet. 
It is an object of the pi-esent invehtion to provide a 
IS system that will allow multimedia messaging via a mul- 
timedia mail box. 

It is another object of the present invention to pro- 
vide a system that is easy to use and which uses access 
tools that are familiar to telephone and Internet users. 
20 tt is a further object of the presehl invention to pro- 
vide a simple visual interface to a message storage sys- 
tem that simplifies the tasks associated with message 
access and administration. 

It is also an object of the present invention to provide 
25 a platform that allows services for a variety of message 
types such as voice mail, video mail and facsimile mail, 
as well as other network services such as Internet and 
intranet services. 

It is another object of the present invention to pro- 
30 vide a system architecture that is easily scaleable. has 
a high availability and which provides a fast response. 

It is an object of the present invention to provide a 
standards based system that will support mailbox ac- 
cess to a multimedia mailbox using conventional web 
35 browser software. 

It is another object of the present invention to pro- 
vide a system in which the message service provider 
does not need to supply the user with any client appli- 
cation software. 

It is a further object of the present invention to pro- 
vide message waiting/urgent notifiers when new or ur- 
gent messages are deposited in the mailbox or the mes- 
sage status changes by a simultaneous different con- 
nection into tfte mailbox such as when a mailbox is ac- 
cessed by computer and while the computer is logged 
into the mailbox an access via a telephone interface de- 
letes a message. 

The above objects can be attained by a system that 
allows a subscriber to access stored messages, such 
as voicemail messages, facsimile messages, e-mail 
messages and video messages, that are stored in a uni- 
fied multimedia mailbox not only through a public 
switched telephone network using a telephone but also 
over a data network, such as the Intern t or an intranet. 
The system provid s voicemail access over th t le- 
phon network, indicating message number, etc. with 
the ability to play messages to the telephone user. For 
text type messages, such as facsimile and e-mail, the 



EP 0 845 Sd4 A2 



4S 



SO 



NSDCCID: <EP 0B45B94Aaj_> 



2 



EP 0 845 894 A2 



system converts the text into speech and plays the 
speech to the telephone user The system allows a per- 
sonal computer user to obtain the data network access 
using an Internet browser. The browser is used to ac- 
cess information about th6 messages stored and is used s 
to download and play the messages via data streaming 
in the case of a voice or video messages or view the 
messages in the case of tdxt type messages, such as 
facsimile and e-mail. The user can also perform the oth- 
er typical messaging functions over the data network io 
connection that are provided for telephone acce^ss, such 
as saving and deleting messages, group list administra- 
tion and other administration tasks. 

These together with other objects and advantages 
vvhich will be subsequently apparent, reside in the de- 
tails of construction and operation as more fully herein^ 
after described and claimed, reference being had to the 
accompanying drawings forming a part hereof, wherein 
tike numerals refer to like pans throughout. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates a virtual unified system unified 
by the actions of a personal computer 60; 
Figure 2 depicts a unified virtual system unified by 
director 80; 

Figure 3 depicts a voice mail system 66 performing 
unification; 

Figure 4 depicts real integrated system 100; 
Figure 5 depicts the functional architecture of a sys- 30 
tem according to the present invention; 
Figure 6 depicts a distributed architecture system 
130 according to the present invention; 
Figure 7 illustrates the flow of control during a re- 
fresh operation; 35 
Figures 6 and 9 depict message list and group list 
templates, respectively 

Figure 10 illustrates the flow of control in a retrieval 
operation; and 

Figure 11 depicts a mossngc retrieval template. 40 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

The present invention provides an integrated multi- 45 
media mailbox and unified messaging. The term "mail- 
box" Is used to mean an entity visible to the subscriber. 
This is the entity the subscriber logs into and appears 
to operate on wrfien the subscriber performs mail-related 
operations. This subscriber-visible entity may not corre- so 
spond directly with a single implementation entity, but 
may exist only through the cooperation of several dis- 
tinct messaging systems, each wich it's own message 
storage capability. To avoid confusion, the term "mail- 
box* is used to mean only th subscrib r visible entrty, ss 
and, where nec ssary. the temn "messag endpoint" is 
used to denote the implementation entity or entities 
which underlie the integrated mailb x. 



Integral d mailboxes have certain d sirabi and 
preferable characteristics A fully integrated mailbox, in 
accordance with the present invention, includes the fol- 
lowing major capabilities that are not present in a single- 
media mailbox: 

a. The ability to deal with messages of different da- 
ta/information types, or having multiple parts (mul- 
timedia mailbox). 

b. A single Inventory (message list), listing all mes- 
sages of all data types, with the ability to control 
presentation of the inventory (e.g., sort the invento- 
ry according to message type, priority, or time of de- 
posit, regardless of the type of message), with con- 
ceptually similar user interface actions for equiva- 
lent operations on any message type, and with the 
ability to randomly select messages for retrieval. 

c. Notification mechanism(s) which can be used to 
alert the user of the deposit of any type of message. 

d. The ability to access the mailbox through a vari- 
ety of commonly-available mailbox access termi- 
nals (PC, DTMF phone, etc.). without special equip- 
ment, and with, as far as practicable, iugicaily th 
same capabilities for all terminal types 

e. The ability to perform data type conversions au- 
tomatically, in support of transparent multi-terminal 
user access, or upon subscriber explicit request 

f. The ability to receive and send messages to sub- 
scribers of existing messaging systems, using a va- 
riety of v^dely-implemented messaging protocols. 

Note that there are degrees of integration in today's 
single-media mailboxes, both with respect of allowed 
message types and the access terminal types which can 
be used. For instance, integrated facsimile/voice mail- 
boxes are common today, and e-mail can be used to 
transfer non-text inforniation. Similariy, e-mail mailbox- 
es cannot be accessed using telephones, and voice/fac- 
simile mailboxes cannot be accessed using a PC. 

Although it is possible to have a mailbox which Is 
integrated with respect to multiple message types but 
which can only be accessed through a single type of ter- 
minal (e.g., e-mail systems using MIME), afully integrat- 
ed mailbox is preferably accessible from several types 
of terminals and pathways, to maximize the subscriber's 
ability to access his messages in various circumstanc- 
es. The following terminal types are provided by the 
present invention; a. Conventional DTMF telephone 
handset; and b. Personal Computer (PC). 

However, other terminal types such as personal dig- 
ital assistants, cellular telephones, two way pagers, etc, 
can be used. 

It should be emphasized that an integrated mailbox 
subscriber using the present invention is able to dynam- 
ically change the terminal used, from s ssiontos ssion. 

The integral d messaging syst m (IMS) of the 
pr sent invention is preferably interfaced to ext rnal 
systems. This allows the subscriber to exchange mes- 
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sages wich external subscribers and can be used to in- 
tegrate several existing messaging accourits on differ- 
ent systems so that the user accesses a single (virtual) 
integrated mailbox. The following types of external sys- 
tems can be included; 

a. The Internet. 

b. Commercial subscription mail systems (usually 
X.400). 

c. Private mail systems (e.g.. MS Mail. cc:Mail). 

d. CFE voicemail systems and other foreign net- 
work-based voicemall systems (e.g., the subscrib- 
er's cellular phone voice mailbox). 

There are several ways, in accordance with the 
present invention, in which integrated messaging can 
be realized. However, the preferred approaches are dis- 
cussed below. 

The integration of the rttailbox can be real or virtual. 
"Real" mailbox integratbh means that the messages of 
all types are located in a single messaging system (MS), 
and that subscriber and administrative control facilities 
for messages and mailbox configuration parameters are 
provided at a single user interface point and do not in- 
volve cooperation or interaction with any other MS. "Vir- 
tual" integrated mailboxes provide the same subscriber- 
visible functionality, and appear the same to the sub- 
scriber as a real integrated mailbox. However, in the vir- 
tual integrated mailbox, the subscriber's messages are 
stored in at least two different MSs. whose configuration 
can be (but need not be) performed separately. The dif- 
ferent messaging systems cooperate to provide the 
complete functionality. The term 'associated MS" is 
used to denote and MS that is in a special relationship 
with another MS for the purposes of synthesizing a vir- 
tual IMS, and the term "external MS" is used to denote 
an MS which is not so closely associated, but whith still 
has an interface to the IMS. 

The distinction between real and virtual integrated 
mailboxes is invisible to the subscriber. Real messaging 
systems can comprise multiple subsystems, such as the 
preferred distributed system described herein, with the 
"mailbox" spread across several pieces of hardware. 
Both types of integration need interfaced to external 
MSs, even if they are not part of a virtual IMS. The re- 
lationship between the MSs that are being integrated in- 
to a virtual IMS ("associated MSs") is much closer than 
that between MSs that just happen to interwork ("exter- 
nal MSs"). While an integrated mailbox system could be 
totally self-contained (allowing messaging only between 
its subscribers, like most voicemail systems today), it is 
preferable to be able to send and receive mail from other 
systems. Real mailbox integration is preferred and de- 
scribed in detail herein. 

Sev ral approach sar available to cr at a virtual 
IMS and are d scribed here: a. desktop int gration, b. 
front-end director, and c. pass-through integration. 

Desktop integration (DTI) is the emulation, as p r- 



ceived by the subscriber, of an integrated mailbox, when 
a personal computer 60 (PC), as illustrated in figur 1 
is used for message endpoint access, there are several 
non-integrated messaging systems (MS) such as e-mail 

s system (EMS) 66 and voicemail (VMS) 68. and the in- 
tegration of the mailbox may not occur foi^ non-PC ac- 
cess terminals; in other woi-ds, special-purpose PC soft- 
ware "does" the integration of the mailboxes; the MSs 
need no ability to handle non-native data or communi- 

^0 cate with each other. One approach to this integration 
is to use a con\^entional browser with a local home page 
that includes link to the various messaging systems. An- 
other simple DTI approach is to use separate TCP/IP 
connections 62 arid 64 to each MS 86 end 68 oh a single 

'5 dialup point-to-point protocol (PPP) connection to a 
router 70 providing hardware dedicated to routihg IP 
packets over various physical hardware interfaces, as 
shown in figure 1 . 

DTI of this type is useful when no close integration 

*o of the MSs is possible for political or administrative rea- 
sons. 

In another approach to virtual mailbox integration, 
a front-end director 80, as illustrated in figure 2 directly 
interfaces with the customer, thus avoiding changes in 

s any of the integrated MSs 66 and 68. The director 80 
communicates on the back end with the separate MSs 
66 and 68 that need to be integrated. Two major variants 
of this approach are possible: the director 80 simply 
passes requests through in real-time, and thus stores 

0 no messages itself; or. the MSs 66 and 68 fonward mes- 
sages to the director 80 when they are deposited, and 
the director 80 stores them until the subscriber logs in. 
In the latter case, the director 80 effectively becomes a 
full IMS with external MS interfaces. 

5 The director 80, if it does not store messages, ac- 
cepts subscriber requests, interprets them, and then 
communicates with the various MSs as needed to either 
retrieve or send messages and inventories. The director 
80 in this embodiment is essentially a transport-level or 

' application-level relay, with some firewall-like functran- 
ality for security. The back end network is typically a fo- 
cal area network (LAN) or high speed wide area networit 
(WAN), onto which subscriber requests are multiplexed. 
However, to provide DTMF access, the front-end di- 

* rector 80 has the necessary voice, facsimile, data and 
voice/data telephone interfaces of a system such as that 
described in U.S. Patent 5. 1 93. 1 1 0. In practice the voice 
user interface is a superset of the interface provided by 
the VMS 68 alone. 

' Pass-through integration is another approach 
where the front-end director 80 functionality is tightly in- 
corporated into one of the MSs. such as VMS 68. The 
MS deals with the messages of its native type, but acts 
as a real-time proxy for subscriber requests for other 

' m ssages types. As discussed previously, the need for 
both voice and facsimile as well as PC acc ss, the dif- 
ficulty of interfacing in real-time to an external VMS. the 
difficulty of augmenting an EMS to handle voice, and the 



NSDOCID: <EP 084SB94A2J_> 



4 



EP 0 845 894 A2 



8 



need for VMS-like notification mechanisms results in a 
preference that the director be added to a VMS 68 rather 
than to an EMS 66 as illustrated in figure 3. 

The foregoihg discussion indicates that ihe prefer- 
able approaches to an integrated multimedia mailbox 
with both DTMF and PC access are either a full, real IM 
system, or enhancement of a VMS so that it can provide 
pass-though r6al-time access to other messaging sys- 
tems. As a result, there are two preferred system-level 
architectures: a. An enhanced VMS (i.e., the IMS) which 
provides all message storage and all user interfaces for 
all types of message. All other MSs interface to the IMS 
as external, non-integrated systems, b. An IMS which 
provides permanent storage for voice, video, text, e-mail 
and facsimile messages and exchanges other message 
types on demand with one or more closely associated 
systems such as an associated EMS (in addition to any 
interfaces to other external MSs). The IMS as all user 
interfaces and passes through user commamds related 
to the associated EMS(s). The IMS exchanges deposit 
notifications with the associated EMS(s). 

Figure 4 illustrates the system/network architecture 
for either the real or virtual integrated mailbox and inte- 
grated message system (IMS). Asiillustrated. the sys- 
tem architecture 1 00 albws access by a telephone 102 
through a public switched telephone network (PSTN) 
104 to the integrated messaging system 106 either di- 
rectly or through a modem 108. Access by a personal 
computer lOg can also be accomplished through the 
PSTN 1 04. Access by a personal computer 110 through 
the Internet 112 using a modem 114 is also provided. 
Note the associated e-mail system (EMS) 115 is shown 
to depict the architecture of a virtual system. The IMS 
1 06 is also coupled to other systems 1 1 6. 1 1 9 and 1 20. 
The content of the IMS 106 will be discussed in detail 
with respect to figure 6. 

The system 100 can also be provided with a multiple 
integrations unit (MIU) subsystem (not shown) such as 
described in U.S. Patent 5,260.990. The MIU can host 
small numbers of PC sessions. This is, however, not a 
large-scale architecture. 

The present invention has the ability to receive, 
send, and store messages of several data types includ- 
ing voice, facsimile, multipart [i.e., voice-annotated fac- 
simile), video, text and e-mail messages. The platform 
1 32 (see figure 6) is designed to accommodate the mas- 
sive amounts of storage required for video data. 

All messages have certain information (the mes- 
sage envelope) that goes along with them, such as 
sender, date/time of deposit, length, etc. The informa- 
tion varies with message type and, to some degree, with 
the means by which the message was received. The en- 
velope information is preferably stored with the messag- 
s carried along with the message if it is to be delivered 
to an external syst m, and be presented to the subscrib- 
r Th IMS 106. as previously discuss d. is able to 
present a single list or inv ntory, containing all messag- 
es of all types (sort d into types), to the subscriber when 



h logs into his mailbox, and provide the ability to select 
messages for retrieval In addition, some of the mes- 
sage envelope information can be presented in the in- 
ventory. The amount of information presented in the in- 

5 ventory, and the format of presentation are determined 
largely by the human aspects of the access terminal; 
when the voice interface is used, the presentation is 
preferably limited to simple spoken message counts 
(■You have three new voice messages, one new facsim- 

10 iie message, and two new E-mail messages and one 
new video message. One voice message is urgent.'), 
otherwise the subscriber may quickly get confused. For 
the same reason, complex inventory sorting, messag 
selection or folder capabilities are preferably not provid- 

is ed through the voice interfate, even though they can be 
if desired. However, a PC interface preferably shows 
much more information to the user without overk>ading 
the subscriber, and allows sophisticated operations 
such as organizing messages into folders. 

20 Generation of integrated message inventories oc- 
curs naturally on a real IMS. A virtual IMS needs to get 
the individual inventories from the external systems 
specified for the subscriber. This is considered below in 
the discussion of interfaces to other MSs. 

2S The voice interface of the present invention 
presents a spoken message inventory which giv s 
counts of messages per type, and additional salient in- 
formation such as whether any are urgent. There are 
essentially two folders: new and saved messages, plus 

30 a virtual "wastebaskef which may be emptied (or not) 
at the end of the session. Selection of a message to play 
is predetermined by the system (play only voice and 
text-to-speech type messages), with some limited ad- 
ministrator controls (e.g., play new or saved messag s 

3S first) ; and the user cannot chose to select a specific 
message, other than by skipping f onwards or backwards 
through the messages. 

A PC interface accordir>g to the present inventkMn 
provides an inventory much like the message list of - 

40 niail systems. Typically, it includes, for each message: 
type of message (voice, video, e-mail, facsimile), sub- 
ject (if any), sender, time of deposit, size of messag 
(bytes, pages, seconds, as appropriate), and status 
(new/read, urgent, replied to. forwarded, etc.). 

45 In a PC interface the user clicks an inventory entry 
to select the message to be retrieved; the system re- 
trieves the appropriate message and provides it to th 
user in the proper format. The PC can support folders, 
so the subscriber can organize his messages. This can 

so be done locally at the desktop, by moving messages to 
local file system directories or files. However, forthe pre- 
ferred type of PC interface (e.g., using a standard web 
browser), the folders can be implemented by the IMS 
106. 

55 Message headers pref rably includ important de- 
tails from the message envelope for non-voice/facsimile 
m ssages. The envelope of messages received via e- 
mail can hav a lot more detail than these of voice/fac- 
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simile messages, and are also in text format. Th se type 
envelope elements are parsed by the system 106 and 
spoken in the same way that sender mailbox humber, 
date/time, urgency and other envelope elements are 
handled (i.e.. by concatenating pre-recorded prompt s 
fragments). Preferably, th6 envelope data Is stored In 
the same way, regardless of message type, to output 
the non-voice/facsimile envelope in spoken format. 

One problem area is in identifying the message 
sender to the subscriber. Ideally, the system should w 
speak the sender's name, preferably in the sender's own 
voice. For messages received from a sender on the 
same system, the system can access the sender's narhe 
announcement (or speak the mailbox number if there is 
no name announcement). The same is true of messag- is 
es from other systems connected through digital net- 
working using the Digital Messaging Network Version II 
(DMN 11)"^ system available from Boston Technology 
Inc. of Massachusetts. Digital Messaging Network Ver- 
sion 11 (DMN W)^ conforms to the AMIS-D voice mes- 20 
saging protocol, which passes the name announcement 
of the sender along with the message, using X.400 and 
a real time DMN II protocol to retrieve remote name an- 
nouncements during message addressing between 
systems. 25 

For non-voice messages received from other ven- 
dor's platforms (including X.400 and MIME/SMTP mes- 
sages from the Internet), the sender address informa- 
tion is a character string, and there is no explicitly iden- 
tified name announcement Of course, an X.400 or 30 
MIME voice body part can be included with any mes- 
sage, and so the sending system can send a name an- 
nouncement. However, the receiving system will not 
know ft is the name announcemeni and just play it as 
part of the message body An example of such an ad- 3S 
dress is "billOabc.com". The receiving system 106 
parses this string and speaks in the same way as a mail- 
box number, character by character. Text-to-speech 
could also be used, but there are problems for today's 
text-to-speech technology in handling an infinite variety 40 
of strange words (such as "abc*) In any case, it is often 
clearer to spell out the address when spoken. A pre- 
ferred solution is to use the prompt voice fragment ap- 
proach with additional prompt Iragments for common 
address elements, to output (lor example) the voice 4S 
prompts -b- 'j- T T "af "a" 'b" "c" "dot" "com" for the 
address noted above. This does not require text-to- 
speech capability. 

The need to handle messages from the external 
MSs 116-120 means that some voice messages will be so 
received in encodings other than the native format of the 
platform. Examples would be MIME messages contain- 
ing audio/wav body parts, or WAV files. The present in- 
vention can output this data to the telephone line just as 
easily as the native format using the f re ware software ss 
package SOX. An alternative is to convert everything to 
the native format, but this can cause quality loss if the 
data subsequently needs to be convert d to some other 



format for output over same other interface and is not 
preferred 

For output to a conventional telephone, non-voice 
messages of the text data type are converted to voice, 
or othenvise represented by voice. However, there are 
some structures (such as the MIME multipart/parallell 
that may be too complicated, or not useful, or impractical 
to output to a voice telephone. In such a situation, the 
system should indicate to the subscriber the type of the 
message and other envelope information, and explain 
that it cannot be retrieved from a DTMF telephone using 
a standard Voice message. 

However, a picture-telephone can receive irpage 
type mail messages. 

Recording of a voice, facsimile or voice-annotated 
facsimile message over the voice interface is performed 
in a conventional manner. To generate other data types 
DTMF key entry is used to select from a menu of simple 
parameterized pre-canned messages. For instance, a 
caller can send text messages containing a callback 
phone number to a subscriber's cellular handset or pag- 
er. The present invention provides these capabilities for 
video, text, e-mail rpessages, as well as for other data 
types (facsimile back services are an example of similar 
usage). 

The limitations of the DTMF keypad do restrict the 
ability to address the message to non-numeric address- 
es. In the present invention, however, a group address 
containing non-numeric addresses (defined using the 
PC interface) can be specified through the DTMF inter- 
face. The present invention also allows a reply with a 
voice message to any received message from any send- 
ing address. The system 106 records the voice and de- 
posits it In the local sender's mailbox, or converts it to 
an appropriate MIME. AMIS-D or X.40Q body part using 
DMN II or the functions of a client e-mail reader such as 
MSExchange. cc:Mail or Netscape Internet Mail, and 
sends it to the original message sender's address, if ex- 
ternal. 

Speech-to-text, or voice recognition, is also a 
means to send text messages from a conventional tel- 
ephone and also a way of addressing non-numeric ad- 
dresses (by spelling them out). 

Fonwarding is the copying of a message and the de- 
posit of the copy in a different mailbox, or transmission 
to a third party, Fonvarding of non-voice messages is 
provided by the present invention through both the voice 
interface and the PC interface because when the mes- 
sage does not need to be converted to voice, such as 
for facsimile messages, the message can be sent either 
to another mailbox, or to a subscriber-entered facsimil 
telephone number, without actually retrieving the mes- 
sage. This approach is also used by the present inven- 
tion for oth r data types (vid o), as is the turnaround of 
lies for alternate facsimile/voice use. When the destina- 
tion is not the same IMS the message is format convert- 
ed as needed. Fonvarding to another mailbox on the 
same IMS is implemented in th same way as for voice 
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and facsimtte, and operates independently of data type. 
Fonwarding toa telephone number, when done automat- 
ically by the system, Is considered a form of message 
deposit notification. The sanne mechanisms are used for 
forwarding, so we address It bel w. s 

Facsimile machines may also be used to access the 
system 106 via the dialup voice interface: often, the fac- 
simile machine may actually be a PC funning facsimile 
software. The system of the present invention has fac- 
simile modem capabilities, and handles facsimile mes- io 
sages in a variety of ways, often in conjunction with a 
DTMF telephone for control and listening to voice anno- 
tations. The system stores facsimile messages In their 
native form (tiff) and no conversion is necessary. The 
additional complexities for the IMS relate to the output 
of non-facsimile messages to facsimile machines. To do 
this the system converts text messages and noh-facsim- 
ile image ddta to facsimile format using a function avail- 
able from Natural Microsystems or the public domain 
software pbmpius can be used to convert to/from vari- 
ous picture forms. Exchange of facsimile data with ex- 
ternal systems is performed using both MIME and X.400 
for G3 facsimile body parts. 

For PC access, two physical interface types are pro- 
vided: dialup to the IMS telephone ports, and via the In- 
ternet (or another TCP/IP network). In addition, several 
ways to handle voice messages are provided; purely 
digital, where the voice data is simply transferred like 
any other type of data (such as by using a browser as 
previously discussed), and the PC turns it intoaudro: the 
use of a voice^data line-sharing scheme, such as pro- 
vided with the Voice View system available from Radish 
(the latter would only be available through the IMS dial- 
up ports) and transfer of the voice data via an e-mail 
attachment with the conversion to audio occurring in the 
PC. 

Accommodating access from a PC (see figure 6) 
can be provided through an APU 150 and IPU 146 (NIU) 
hosting SMTP/MIME and POP protocols. Because of 
the rapid growth in Internet connectivity and on-line 
services, message user agents (MUAs) (or client mall 
readers) using these protocols are becoming universal 
for subscribers using dialup access. However, the Hy- 
pertext Transfer Protocol (HTTP) is the preferred proto- 
col since it is the one used to interface with standard 
web browsers. It is a simple request/response protocol 
which uses TCP/IP. Requests (called methods) are pro- 
vided to get and create objects (real or synthesized da- 
ta), and to do other operations In support of navigating 
a global, interconnected set of information. The subjects 
of the methods are identified by a Universal Resource 
Identifier (URI) or Locator (URL) which specifies the lo- 
cation (including the Internet name of the host vvhere the 
information is stored) and the means to access the ob- 
ject. Responses are r tumed to a request r in MIME- 
compatible format, so the MIME content-type and con- 
tent-encoding can b d termined by the requester, and 
the object presented in the appropriate way. To actually 



provide the "web" of connected objects, specially-for- 
matted text scripts or templates, using the Hypertext 
Markup Language (HTML), are used. These "web pag- 
es" can have embedded links to other objects systems/ 
hosts), as well as presentation control capabilities. The 
browser follows these links by using the linked object's 
URI to send a GET request to the host identified in the 
URI, when the user selects the URI. By putting links to 
further pages (HTML static documents, or dynamically 
synthesized documents) in a web page, a hierarchical 
organization of infonnation can be established. HTML 
and HTTP also support entry of data via forms, access 
to files via FTP, and interfaces to other information sys- 
tems, such as e-mail, GOPHER, and News. 

To provide the PC 1 09 or 1 i 0 (see figure 4) with web 
browser access, the IMS 106 of the present invention 
provides an HTTP server (IPU 146) to handle the re- 
quests from the browser, and provides an organization 
of the information in the IMS 1 06 into a logical structure, 
using HTML pages as will be discussed in nnore detail 
later. 

In general, HTTP servers are widely available, both 
as public domain source code, and as commercial prod- 
ucts. Toolkits are available to simplify page creation. HT- 
TP servers are highly configurable, including the map- 
ping of universal resource locators (URLs) to internal (or 
external) data or operations (executable scripts). 

A simplified example of an HTML page hierarchy 
for the messages stored in the IMS and the subscriber 
operations used to access messages is given below. 
Sample URLs are given to aid understanding. 

a. Subscriber (or caller) uses a browser 144 (se 
figure 6) on a PC 142 to open subscriber's IMS 
home page (e.g.. "http7/www. mail. some rboc.com ./ 
JoeQUser/". or "http://www.maiLsomert>oc.com/ 
61 7-2461^9000/") or http://www.mail.some rt)oc.conrV 
awscripts/btv.dll?REFRESH. A "bookmark" could 
also be used to remember the URL. 

b. The PC 142 dials directly to the IMS 132 or via 
an Internet service provider (ISP, 140, arxj connects 
to the IMS host 132. The PC 142 sends an HTTP 
GET method for the home page URI. 

c. The IMS HTTP server 140 sends the home page 
in the response to the GET and the PC's browser 
1 44 displays it. The home page has welcome text 
and/or graphics and/or voice announcement, in- 
cluding a password entry field. For users wishing to 
leave a message for the subscriber instead of ktg- 
ging in, there is preferably a "button" on the sub- 
scriber's home page to "leave a message." The call- 
ing line identifier (CLI) can be used to verify that the 
calling number matches the subscriber name, or for 
using the authentication capabilities of HTTP. A se- 
cure sock t layer (SSL) is used to provkfe an initial 
s cure connection for authentication. A menu of ad- 
ditional services besides integrated messaging can 
also be provided. 
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d. Using the browser 144. the subscriber enters the 
password in the login information fornn, the browser 
144 sends it to the HTTP server 146. The server 
146 validates the password and synthesizes the 
subscriber's main messaging page ("http://wWw. 
maiLsomerboccom/JoeQUser/ ihbdx") frorti the 
contents of the subscriber's message store, an6 re- 
turns the page to the browser 1 44. The subscriber's 
messaging page contains links to each of his stored 
messages ("httpV/www. mail, somerboc.com/aw- 
scripts/btv.dll?REFRESH) and whatever inventory 
information ("http://www.mail.somerboc, com/aw- 
scrlpts/btv.dll?DF^TR&Unique Msgid) is desired for 
display, plus buttons for sending, deleting, forward- 
ing or other message actions. 

e. If the user clicks a message link, the message is 
retrieved by the HTTP server 145 from wherever it 
is stored. The message is formatted and encoded 
as a t\A\ME nnessag6, and is presehted by the 
browser 144 according to the MIME type (text, im- 
age, video, voice, etc.). Voice and facsimile mes- 
sages stored need to be converted to M\ME format 
first. 

f. Buttons are also links to URIs. If the subscriber 
clicks a button, the HTTP server 146 may simply 
use the URI as a conirri&nd name and perfomn the 
action, or it may return an HTML form to allow fur- 
ther user input, or to give more details about some 
entity. 

In the present invention, an HTTP server is provided 
in each application processing unit (APU) 1 50 (see fig- 
ure 6) that is configured to handle data calls. In addition, 
an HTTP server (IPU 146) is provided for subscribers 
accessing the II^S 132 via an ISP 146 and the Internet 
1 36. For small numbers of concurrent PC sessions, the 
MIU previously mentioned, with modem banks, can also 
be used to provide PC dial-up access. Note that all HT- 
TP servers are configured identically, and use the dis- 
tributed subscriber database access mechanisms (in 
the same way as a voice application) to locate and re- 
trieve subscriber data and messages, even when they 
are not on the same APU. The HTTP servers use the 
existing platform operating system TCP/IP and PPP 
protocol capabilities. 

Two situations for data type conversion can arise: 
when a subscriber's terminal type will not accept the 
stored message formal, and upon user request. An in- 
termediate situation is when a subscriber requests de- 
livery or fonwarding of the message to a system or ter- 
minal, other than the one he is using, which does not 
support the data type. Most conversions are implicit 
from the message type and the destination, but it may 
be pr ferabi for a subscrib r to xplicitly r quest con- 
versi n (e. g. ) of a facsimile message to text for forward- 
ing to an Intern t address, even though the message 
could have been sent as a MIME facsimile message. 

For the IMS of the present invention, messages are 



preferably stored either in native voice format as de- 
scribed in U.S. Patents 5.029, 199 and5. 193.110. native 
facsimile iormat (preferably conventional tiff), or in a 
MIME format. For PC access, or delivery to a PC via 
s outdial, handling of data other than native Voice data is 
preferably by sending it unconverted using the stored 
1^1 ME format. Facsimile information is preferably sent 
in the MIME image/tiff fontiat. Voice data is preferably 
sent as the MIME audio/wav type. 
10 Notifications are used primarily to alert a subscriber 
that he has messages, so the subscriber ne6d only ac- 
cess the system when messages exist. Notification 
mechanisms vary from the communication of one bit of 
infornhation (messages present or not), to delivery of the 

IS actual messages themselves. 

The present invention, using Access NP'*", pro- 
vides comprehensive notification delivery capabilities 
including: paging via outdial. TAP andTNPP; message 
waiting indication (MWI). via SMDI. and Using various 

20 SS7 or ISDN capabilities; special delivery outdial (deliv- 
ery of voice and facsimile messages by outdialing the 
subscriber at a specified telephone number); and cellu- 
lar short messages, containing mailbox message 
counts or callback numbers. 

2^ Most of the above do not actually deliver messages, 
and so require no message datatype conversions. Spe- 
cial delivery, however, is more complicated. The desti- 
nation tenrninal must be capable of receiving the data 
type, or the system must be capable of recognizing th 

30 terminal type and converting the message accordingly 
The data conversions discussed (e.g., text-to-facsimile 
or text-to-speech) should, however, handle most notifi- 
cations. Special delivery via outdial to a PC is also pro- 
vided. 

3S In addition to data conversion for special delivery, it 
is necessary to consider how other message types af- 
fect the notification algorithms. One approach is to han- 
dle them in exactly the same way, so that any e-mail 
message causes the MWI light of the subscribers tele- 

40 phone to come on, or causes a page to a pager if it is 
marked urgent. 

A real IMS is self-contained for notifications, in that 
any message which is to be notified is stored in the IMS. 
However, in a virtual IMS, one MS does not store all 

4S messages. Further, some of the integrated message 
systems may have no notification capability. Thus, there 
needs to be a way to communicate the presence of mes- 
sages between MSs ("cross-notification"). This may re- 
quire non-standard mechanisms, since most MSs are 

so not designed to accept notifications. This, in turn, may 
require MS modifications, which defeats much of th 
purpose of a virtual IMS. Below we discuss how to pro- 
vide VMS exchange notifications with an associated 
EMS, without major EMS modifications. 

55 An associated e-mail system (EMS) will preferably 
have one of SMTP/MIME and POP protocol capabilities 
(or quivalent X.400 capabilities). The EMS can initiate 
SMTP sessions, but the VMS must initiate POP ses- 
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sions. The EMS is configured to automatically create a 
copy of each message deposited, or an additional mes- 
sage with inventory information, give the message a 
special VMS recipient address, and send it via SMTP to 
the VMS (IMS 106). This has desired asynchronous no- 
tification characteristics. The VMS (IMS 106) receives 
the message, parses it (or merely notices It), and uses 
the information to control the notificatbn rtiechanisms 
(but not score it). Alternatively and less preferably, the 
VMS (IMS 106) uses POP to poll the subscriber's mes- 
sage inventory. This is a periodic event, likely to cause 
a lot of traffic, since it needs to be fairly frequent and 
there could be large numbers of subscribers. In either 
situation, the VMS (IMS 106) needs to be provided with 
the subscriber's e-mail address and password. 

There Is also a hybrid of the aboVe-didcussed ap- 
proaches where the VMS is an IMS, and it uses POP to 
retrieve EMS messages but also offers POP to the sub- 
scriber so that client e-nr>ail SW connects to the VMS 
(IMS) to retrieve e-mail messages, 

Fornotificationsf rom the VMS to the EMS. the VMS 
sends an e-mail text message containing the voice/fac- 
simile message inventory using SMTP. 

For both real and virtual IMSs, there is a need to 
interface with external systems 116-120 using the reg- 
ular store^^and-forward e-mail paradigm, in order to 
make the IMS subscribers part of a single world-wide 
messaging community. For the virtual IMS, special con- 
sideration must be given to the "associated MSs", since 
that non-standard (for e-mail systems) but conventional 
techniques need to be applied. 

The Access NP™ platform available from Boston 
Technology, Inc. provides AMIS-D digital) and AMIS-A 
(analog) networking for transferring messages to other 
VMSs with the requisite protocol support. This includes 
most VMS vendors. The system handles voice data ac- 
cording to the AMIS-D specification. It also has down- 
ward-compatible enhancements for interworking with 
other systems, so that multiple Boston Technology IMS 
clusters become essentially connected in near real- 
time. 

For interfacing with external VMSs, or between Bot- 
ton Technology, Inc. IMS platfonms, AMIS-D is pre- 
ferred. The X400 technology is also usable for inter- 
working with other X.400 mail systems, allowing con- 
n ctivity through X.400 administrations to many e-mail 
users. Since almost all X.400 public mail systems have 
Internet connectivity, this is a preferred approach to pro- 
viding world-wide connectivity for IMS subscribers. 

When external MSs do not have X.400 interfaces, 
MIME/SMTP protocols are the next most preferred 
choice, both for Internet connectivity and for connection 
to corporate LAN mail gateways. 

For a virtual integrated messaging system (IMS), 
some of the systems outsid of the VMS 68 need special 
treatment, so that the m ssage endpoints of th MS (for 
exannple, e-mail system 66) appear to be part of the 
same integrated mailbox as the VMS message end- 



point. Interfac s and operation for the "pass-through di- 
rector" approach to virtual IMSs will be discussed in 
more detail (notifications have been addressed above 
already). As for notifications, an EMS with SMTP and 
s POP capabilities, connected as shown in figure 3, is as- 
sumed. 

The general handling of a virtual IMS subscriber's 
operations in a virtual IMS will be described below for 
each of the typical phases of a session. 

10 When the subscriber logs into the VMS 68, the VMS 
68 generates an integrated message inventory. This is 
done through the use of a POP session between the 
VMS 68 and the EMS(s) 66. The VMS 68 logs into the 
POP server on behalf of the subscriber (POP USER and 

IS PASS commands), queries the EMS inventory (POP 
LIST command) and merges it with the local inventory. 
The VMS 68 maintains a map from the POP message 
IDs to an identification (place in sequence. ID number 
or synthesized URI) used between the VMS 66 and the 

20 subscriber Locally stored messages are likewise given 
identifiers. 

To select a message for retrieval, the subscriber in- 
terface may use POP (for MUA interfaces), an HTTP 
GET method (for web browser interface access), or en- 

25 ter DTMF commands (for voice access). In any event, 
the VMS 68 maps the subscriber-requested messag 
ID or URI to the EMS message ID. or to an intemal mes- 
sage ID. If the message is on the EMS 66, the EMS 66 
uses POP to retrieve the message from the VMS 68 

30 (POP RETR <msg id> command), and outputs it to the 
subscriber If the message resides on the VMS 68, it is 
simply retrieved and played. Data conversions, if nec- 
essary, are performed as described earlier. 

To submit a message, the VMS 68 needs to deter- 

-35 mine whether the message is to be sent to the EMS 66 
or handled by the VMS 68. If the former, then the DMN 
II system previously mentioned (using the X400 proto- 
col) is used to forward the message to the EMS 66. The 
criteria for the decision would typically include the recip- 

40 lent address and the message data type. There are 
many possibilities for the algorithm; the preferred is: 

a. If the message is native voice or facsimile (i.e.. 
using the DTMF interface), and the recipient ad- 

45 dress is a phone number, the message is handled 
in the conventional VMS manner for a voice or fac- 
simile message. 

b. If the message is native voice or facsimile, and 
the recipient address is not a phone number, the 

50 message is sent to the EMS 66, with the data con- 
verted to a MIME audio or imageAiff type. 

c. If the message is a MIME audio or facsimile type 
(i.e., SMTP or HTTP user interface), and the recip- 
ient address is a phone number, the data is convert- 

55 d to nativ format and handled as conventionally 

by the VMS (note: addressing to a phone number 
could be prohibited to avoid this conversion, initial- 
ly)- 
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d. If the message is any MIME type, and the recip- 
ient address is not a phoiie numbfer. the message 
Is sent unchanged to the EMS 66. 

e. When messages are sent to the EMS 66 for han- 
dling, the sender address is set to ths e-mail user 
name of the subscriber, so it appears that it origi- 
nated in the EMS 66. 

The approach discussed above Is a straight forward 
way of operating. There are nnuch more complicated 
ways of doing it. The advantage of the simple approach 
is that the VMS involvement In handling e-mail messag- 
es is minimized. 

Figure 6 Illustrates a general functional architecture 
for a real IMS which provides the capabilities discussed 
above. A more detailed description will be provided With 
respect to figure 1. 

Various elements of the archlctecture will be dis- 
cussed. Note that figure 6 Is not Interhded to be a defin- 
itive and complete representation of the actual software 
architecture of the IMS which will be described in more 
detail with respect to figure 6. 

The dialup interfaces 130 In the APUs 150 have 
drivers for access to conventional DSP hardware, pro- 
viding voice interface functions (V), facsimile modem 
capabilities (F), data modem capabilities (D) iand 
VotceView voice/data modem capabilities (V/D). 

For DTMF access, the voice application 131 is the 
application described In the related patent previously 
mentioned, which accommodates multimedia messag- 
es and data conversions, for example text-to-speech 
and text-to-facsimile. 

The multimedia message store 132 Is distributed 
across all the APUs 150. with most of a subscriber's 
messages being stored on a "home" APU. Standard 
methods are used by applications in any subsystem to 
access the distributed message store in a location-inde- 
pendent way. 

The Boston Technology, Inc. Digital Messaging Net- 
work Version II (DMN II)™ product 133 provides store- 
and-forward communication with other MSs, including 
other Boston Technology I MSs or VMSs, other vendor's 
VMSs, and external EMSs. Either the AMIS-D protocol 
1 34 or SMTP 1 35 Is used for this. DMN tl Is automatically 
activated when a message requires delivery to an ex- 
ternal address. 

The subscriber PC access functions reside in the 
APU 150 for dialup access, and in the NIU (IPU 146) for 
n twork access. Both instances of this set of functions 
are essentially the same, except that the datalink proto- 
cols are different. 

An HTTP sen/er 136 is provided in the IPU 166 to 
allow access using conventional web browsers and as- 
sociated web pag s 137, and SMTP and POP serv rs 
1 38 are provided to allow access from MUAs. All other 
services POP3/IMAP4A/oice/PPP/X.400 are utiliz d on 
other components and in turn utilize the IPU 1 66 for HT- 
TP if necessary. 



The PC application 1 40 provides the structure of the 
user interface for HTTP access, and th necessary 
"glue* to interface with the internal system mechanisms 
(in this case. jUst the message storage). 

s Microsoft WinSock 1 42 is us6d to provide the TCP/ 
IP protocol support, both externally and internally Win- 
Sock Is an integral part of Windows NT, and includes 
support for Voic6View data transfers. For Boston Tech- 
nology, Inc. platforms using the UnixWare OS, the TCP/ 

10 IP support is also part of the OS, but VoiceView is not 
supported. 

The "real" IMS system 130 (106) of the present In- 
vention Id preferably implemented using a distributed ar- 
chitecture such as illustratCMd in figure 6. The system 1 30 

1^ includes a message platform 132 that Is connected to 
both the public switched telephone network 134 (via a 
digital matrix switch 135) and the Internet 136. A sub- 
scriber can access the platform 132 using a telephone 
1 38 to perform message access functions such as re- 

20 trieving and listening to voice mail messages, forward- 
ing messages, recording messages, and converting and 
playing facsimile messages. A detailed description of 
this type of access can be found in the U.S. patents pre- 
viously described and is available in the GO Access® 

2S and Access NP™ systems from Boston Technolohy, Inc. 
The subscriber can also access the platform 1 32 for ac- 
cessing voice mail messages and other types of mes- 
sages such as facsimile and video messages over the 
Internet 1 36 through a conventional Internet service pro- 

30 vider system (ISP) 1 40 using a personal computer 1 42. 
The personal computer 142 is a typical convention- 
al multimedia personal computer capable of running or 
executing an Internet browser 144, such as the pre- 
ferred Netscape 3.0 browser available from Netscape 

3S Communications or Internet Explorer 3.0 available from 
Microsoft, and capable of connecting to an Internet serv- 
ice provider 140. The PC 142 can also, of course, be 
connected to the Internet through a company local area 
network (LAN) via a high speed connection. The com- 

40 puter 142 preferably includes a modem with a speed of 
at least 14.4 Kbps and preferably at least 28.8 Kbps 
when the user intends to access video images. The 
computer 142 also includes a conventional sound card 
and associated audio speaker and audio software such 

^ as the preferred TrueSpeech available from the DSP 
Group. For the recording of voice messages that will be 
transmitted to other mailtK)xes the computer 142 needs 
a microphone half duplex recording board such as 
SoundBlaster from Creative, Inc. and software such as 

so MediaPlayer from Microsoft. For displaying still images, 
such as facsimile mall, the computer 142 needs to in- 
clude a tiff reader such as Microsoft Facsimile availabi 
from Microsoft Corp. These still image components can 
also be used to record facsimil messages for Iransmis- 

ss sion to other mail boxes. For th playing of motion video 
images, the computer 142 uses ActiveMovl from Mi- 
crosoft. If video images for broadcast are to b recorded 
the computer 1 42 needs a conventional camera and as- 
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sociated software such as Connectix from Connectix. 
Inc. 

The browser 144 is used in a conventional method 
to access an Internet web paQe hosted by the platform 
1 32. The browser 144 allows the us6r to perform func- 
tions such as viewing a rhessage list, view addresses of 
voice/video itiarl recipients and senders as well as e- 
mail addresses, playing and saving messages ^ forward- 
ing messages to others, replying to messages, creating, 
listening to and rhodifying personal greetings, narhe an- 
nouncements and prompts, etc. using the cjraphical user 
interface and the video/sound capabilities of the com- 
puter 142. 

The user accesses the platform 1 32 through one or 
nnore Internet processing units (IPUs) 146 over a con- 
ventional digital communicatioh path typically used for 
high speed access (1 0-100 Mb/s of a home page Server. 
The unit 146 is preferably a pentium based computer 
that operates at 133^166 MHz with 32 MB of RAM and 
4-GB hard disk drives that are mirrored. A conventional 
Internet router (not shown), such as available from Cis- 
co Systems or Bay Networks which performs packet fil- 
tering, can also be provided between the Internet 1 36 
and the unit 1 46. The router can provide a "bridge" be- 
tween the ethemet 149 and the network backbone. In 
such a configuration the router and unit 146 would be 
coupled preferably using a 100Mb Ethemet connection 
and perform the primary function of moving packets to 
and from the IPU 1 46. The unit 1 46 provides firewall pro- 
tection from hackers, acts as a web server for the mes- 
saging application, performs hypertext markup lan- 
guage (HTML) generation and performs the voice en- 
coding and image encoding for the messages streamed 
to the computer 142. 

During a typical session a user will access the plat- 
form 1 32 over the Internet 1 36 using a standard web 
browser to obtain a service provider home page where 
the user will tog into the Internet service provided by the 
platform 132. During this process the user is required to 
enter a mailbox identifier and* a pass code which are 
checked to ensure that the user is authorized. Once au- 
thorization is confirmed a service session is initiated and 
the user is presented a page that includes a menu of 
service options such as viewing a message list, admin- 
istering mail box options, other network service fea- 
tures, etc. 

When the users e-mail is stored and supported by 
another platform the message list can include a cross 
notitication of the existence of an e-mail message in the 
mailbox message list. 

During a typical message retrieval function, the unit 
1 46 accesses a master control unit 1 48 over an internal 
dual channel ethernet 1 49 to locate the storage location 
of the various types of m ssages stored for a subscriber 
and g nerat s a w b pag which is transmitted to the 
personal computer 142 and which includes a list of the 
messages. The user selects a message in a convention- 
al fashion by double clicking on a message descriptor 



or sel cting the message and clicking on an appropriate 
icon such as "Play" in the display of the browser 144. 
The unit 146 responds by obtaining the selected mes- 
sage from the application proc ssing unit 150 that 

s stores the message, cohverting the message irito th 
proper format and transmitting it. In the case of voice 
messages the voice data is converted from the encoding 
for storage into a file in the encoding for playing used 
by a conventional audio application executable by the 

10 browser 1 44 such as the preferred ActiveMovie f rotn Mi- 
crosoft and streamed to th6 browser 144 where it is 
played as it is received. For facsimile and other text mes- 
sages a tiff file i6 created and transmitted to the browser 
144. For video messages the video data, if necessary, 

15 is conversed into the avi, mpg, mpeg, cu, etc. file formats 
that allow the data to be streamed to the browser and 
displayed in a pop-up window in real-time a& it arrives. 
That is, video as well as audio messages are played or 
displayed as received by the computer 1 42. During play 

20 back the user can perform the conventional functions of 
rewinding, pausing, fast-forwarding, skipping, etc. The 
user can also perform operations associated with saving 
the message, deleting it or forwarding it to others. The 
processes performed by the Internet processing unit 

2S 146 as well as those performed by the master control 
unit 148 and application processing units 50 and other 
units 52, will be discussed in nrtore detail later and are 
set forth in the source code appendix which code can 
be stored on/in various types of media, such as various 

30 types of disks and various type of computer memories, 
in the platform 132. 

The operations or entry points available to a user 
interacting with the browser 144 include the following 
functions: 

35 

QUIT - Exits out or togoff of the system and de- 

lete all session information. 

DELE - Removes one or more messages from 

the system. 
40 SAVE - Saves a message. 

DRTR/RMSG - Retrieves data such as an audio mes- 
sage. 

RECORD - Sets and starts record/send p roc ess . 

REFRESH - Gathers data and instantates to brows- 
es er in HTML 

USE - Selects a specific template. 

GRTR - Retrieves particular group list data. 

GDEL - Deletes a group list. 

GINS - Adds a single entry to group list. 

so GUSE - Get group and use template. 

GPUT - Modify current group lists. 

GNEW - Make new list. 

MBOXADM - Administer user mailbox features such 
as change th password. 

55 

Each of these operations has a routine of a correspond- 
ing name in the source code appendix. The prccesses 
for the above-listed functions perform many of the same 
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operations such as auth nticating a request, however, 
for simplicity of explanation three representative proc- 
esses, the process that refreshes (REFRESH) the 
browser display, the processes (DRTR/RMSG) that 
download nnessages for playing/displaying and the s 
process records (RECORD) messages to send to oth- 
ers, will be discussed. The other processes are de- 
scribed in detail within the source code appehdix. 

The refresh operation steps, with the data, ac- 
cessed, are depicted in figure 7. The refresh operation 'o 
is integral to the display update operation of the irtven- 
tion because each of the web pages that are transmitted 
to the computer 42 is affixed with an expiration date, 
such as yesterday, that causes the browser 144 to re- 
quest a reload of the page each time the page is ac- 
cessed or additional information is requested. This re- 
sults in the message list being updated to include mes- 
sages that have arrived since the current session start- 
ed. The refresh operation is also used to provide web 
pages that the user has selected during browsing where 20 
the refreshed page has never been transmitted. The re- 
fresh operation starts with a refresh request (a "GET" 
with a URI) being sent 1 to the Intemet processing unit 
146. The IIS routine 170 denies access if the authenti- 
cation for the request is not present. If the authentication 25 
is present the request is passed 2 to the security filter 
routine 172. The security filter 172 sends 3 the request 
to the support 1 74 for authentication by the platform 1 32. 
The routine 174 checks 4 the session information data 
175 to see if a "session key" currently exists for the re- 30 
quest and if so the flow skips to step 9 discussed below. 
If the session key does not exist the authentication re- 
quest is sent to UNIX support routine 176 of the MGU 
1 48 and/or APU 150 for authentication. The authentica- 
tion involves accessing 6 the user authentication data 35 
178 and returning 7 an indicator of success or failure at 
the authentication task. The authentication successful 
result is returned 8 to the security filter 172 otherwise 
processing proceeds to step 21 where a message con- 
cerning the authentication failure is returned to and dis- 40 
played by the browser 144. The security filler 172 trans- 
fers 9 control to the validation routine 1 80 where account 
and file systems permission validation is performed and 
which returns 10 the result of validation. If validation is 
not successful, control transfers to step 21 where a fail- 45 
ure message is sent to the browser 1 44. If the validation 
is successful control transfers 11 and 1 2 through the Mi- 
crosoft Internet Information Server (IIS) routine 170 to 
the template update routine 182. Routine 182 transfers 
1 3 the control to routine 1 74 to access 1 4 and check the so 
state of the session cache 1 75. Once this check is com- 
pleted a request 1 5 for an update of the message/group 
information is presented to the routine 176. The routine 
176 obtains 16 the mailbox list information or group list 
information 177 stored in the MGU 148 and r turns 17 ss 
the infomnation to th unit 1 46 where the session cache 
is updated. The updated data Is forwarded 1 9 to the 
HTML template update routine 182 where the current 



template from the t mplate file 34 is instantiated 19 to 
match the data in the session cache 175. The data of 
the template is passed 20 to the IIS routine 170 where 
the HTML template for the page is returned 21 to the 
browser 144 which displays the refreshed page to the 
user. The templates preferably use the Microsoft.hlx file 
syntax and the templates include standardized varia- 
bles for the various data, such as <%AccoUntNum- 
ber%>, <%From*yo>. <%f^edia%>, <%Lehgth%>, mc. 

The transmission of the HTML for a template allows 
the service provider to custdhiize the home page (for 
example, by adding advertisements) and deliver pages 
based on the domain or class of service of the subscrib- 
er, 

A refreshed active page that is designed to list 
stored messages can be formatted as illustrated in fig- 
ure 8. The messages can be accessed (played/dis- 
played in real time), saved, deleted from the platform 
database and in saved to storage local to the personal 
computer 1 42 using the conventional point an click par- 
adigm. 

The group list administration functions can be facil- 
itated using an active page such as depicted in figure 9 
where custom distributiin lists can be easily created and 
edited to add, delete and modify addresses of recipients 
of the various type of messages that are supported such 
as voice, video, facsimile, telex, e-mail, etc with ad- 
dresses such as telephone numbers, e-mail addresses, 
universal resource location addresses, cable address- 
es, etc. 

The processing of a message retrieve or play re- 
quest (DRTFVRMSG). like the refresh request, starts 
with the transmission of a request 1 by the browser 144 
to the unit 1 46 as depicted in figure 10. The request can 
be made by double clicking on a message in the list of 
figure 8. The request is processed in the same way as 
th6 refresh request for steps 2-10. In step 11 the name 
of the requested file which has been made using a cod- 
ed name may also be converted to the real file name, 
which is a security issue which will be discussed in more 
detail later. The file request is passed 12 to the gateway 
routine 182. The routine 182 passes the request for the 
file to the support routine 1 74 which obtains 1 4 a unique 
identifier from the session information 175. The request 
is then provided 15 to the application processing unit 
150. The processing unit 150 accesses 16 the list of 
messages 177 and obtains 16.5 the message data 186 
(voice, text, video, etc. ) from storage. The message data 
passed to the support routine 1 74 and is converted from 
the native storage format (from Oki 24 when the mes- 
sage is a voice message) into and stored 1 9.5 in cache 
or temporary storage 188. The cache storage allows lat- 
er requests for the same message to be processed with- 
out again decompr ssing the data. As th data is tem- 
porarily stor ditisretri vedasneed dtop rform a real- 
time conversion into str aming mod data and into th 
desired format, such as TrueSpeech. This conversion 
will conv rt voice data into data compatible with Net- 
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scape type "plug-in" play back systems that Include 
TrueSpeech. Vomare. Real Audio and WAV (which is 
Standard for Winddws 95/NT). If the data is text data it 
can be converted into a tiff file compatible with a con- 
ventional tiff viewer A facsimile message is typically 
stored in a tiff file. If the data is video data it is converted 
into MediaPlayer data (an .avi tile) compatible with the 
MediaPlayer system available from l^icrosofl. The data 
can also be converted into JPEG or MPEG or still and 
motion video players available for conventional brows- 
ers. The real-lime streamed data, which includes the 
content type, is sent 18 to the IIS routine 170 and im- 
nnediately forwarded 19 on to the browser 144 with an 
option to store the data in thd cache local to the unit 146. 
When streamed the browser 1 44 does not know the con- 
tent type of the rtiessage until the message is received 
from the server 146 and thd invention relies on the de- 
fault mode of the plug-in that handles the data type. At 
the browser 144 the transnnllted data, if It is Voxware 
data, causes a window to open and it is immediately 
played or displayed as the case may be. Other types of 
data such as RealAudio require that a play button in a 
pop-up window be activated. 

Once a message is received the subscriber, using 
the browser 144, can save it locally, play it again, re- 
verse it, skip forward and backward, copy it into other 
f i les and other types of operations that can be perf onmed 
with multimedia files. 

During a record process, a media player, delivered 
with a conventional operating system such as windows 
95/NT, records the message or the subscriber activates 
a plug-in of the browser 144 to record the message 
(voice, video, text. etc.). Using the application, the sub- 
scriber edits, rerecords, etc, the message until satisfied. 
When finished the user provides a file name to the re- 
cording and stores it locally. The browser 144 is then 
used to POST the message to the server. Alternatively, 
the browser 1 44 can be used to request that the server 
146 record the message and send it. The server re- 
sponds by perfomning the steps 1-21 for refreshing a 
page previously discussed and forwards a template for 
a page, such as depicted in figure 11. The subscriber 
completes the template by providing the file name of the 
recording, addresses (such as telephone numbers, e- 
mail addresses, etc.) of the recipient along with indica- 
tors for privacy, etc. The browser 144 when the submit 
message" button Is pressed creates a message header, 
attaches the file and sends the file to the server 1 46. 
The server 146, when the file appears In the incoming 
message directory, converts the message into an ap- 
propriate storage format (compresses it if necessary) 
and stores the message. The header is reviewed to as- 
certain the recipients addresses and the message is re- 
trieved and sent to th recipients. For example, a voice 
message to a particular t lephone number would result 
in an outdial process being performed. If the recipi nt is 
a subscriber the messag is copied to the recipients 
mailbox. 



The present invention enhances th security of 
each session wilh a number of different features 

Unit 126 is preferably limited to the HTTP (Hyper- 
text Transfer Protocol) and SSL (Secure Sockets Layer) 

5 type Internet service to help eliminate problems associ- 
ated with unsecure protcwrols. The present invention al- 
so has certain authentication features. A request is first 
sent frorin the browser 144 to the server 146 that does 
not include authentication. The Microsoft Internet Infor- 

10 matk>n Sen/er (IIS-170) software running in the sender 
146 checks this request. The fitter 170 also checks to 
. see if the user is valid and since there is no authentica- 
tion the check fails and the browser 144 sends the re- 
quest again this time with authentication. The request 

IS either passes or fails based on the authentication. When 
the request fails the browser 144 is notified. When the 
request passes the request is forwarded for processing. 
In an initial log- in situation the request is fonwarded to 
the users home APU 150 where the validity of the user 

20 is checked again to determine whether the user is a sub- 
scriber. This double validation helps prevent nonsub- 
scribers from obtaining access to the system. When the 
user is a subscriber the user gets a message list sent to 
the browser 144 where it is displayed. Once a session 

2S is established the user it essentially communicating with 
the home APU 1 50 for this subscriber which controls fur- 
ther transactions, AM further requests by the browser 
144 to the server 146 go through the first level of au- 
thentication. 

30 The system preferably uses secure socket lay r 
(SSL) packet encryption. 

The present invention also is preferably implement- 
ed using dual homed-host Internet processing units that 
prevents packet sniffing on the internal ethernet. A dual 

3S homed-host is a host that has two I P addresses that cor- 
respond to one or more physical addresses altowing it 
to be configured differently based on the IP address. For 
example, one IP could be configured only to work with 
SSL active and the other IP is used in the clear, i.e. with- 

40 out SSL 

The provision of a router to perform packet filtering 
prevents source address spoofing. 

The present invention also assigns session num- 
bers and specially created file names to files that are 

4S transferred to the browser 44. In this filter operation th 
process removes all correlation to any data set internal 
to the platfonm 1 32 from data sent over the network 1 36. 
For example, a message identifier that is sent to the 
browser includes a session identifier and a randomly as- 

so signed file identifier (which can be the current time of 
day). The server 1 46 creates a session information entry 
that identifies the file for the session identifier and th 
randomly assigned file identifier. When the browser 144 
r qu sis th file the s ssion identifier and previously as- 

55 signed file identifier ar included with the r quest. The 
serv r 1 46 uses the session information to conv rt the 
file name into a real file name to retri ve the file. When 
the server 148 s nds the requested file to the browser 
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1 44 the server 146, if it is not a streamed data file, the 
server 146 assigns a pseudo file name that includes a 
session number and a randomly created file name. This 
pseudo file name and the real file name are also stored 
in the session information so that the fil6 can be request- 
ed again. The session mumber is part of the create Iden- 
tifier to allow communicailoh that use the same random 
n umber to be distinguished. If the file is a streamed data 
file, the data of the file With corltent type Is sent without 
any file name identification. 

The present Invention preferably uses a buffer of 
8 1 92 bytes to improve transfer efficiency even though a 
buffer size of from 512 byters to 8192. except for 4096 
for certain audio formats when the data is audio, will 
work. 

The present invention, through the network inter- 
face provides two methods of message deletion: imme- 
diate or marking messages to be deleted with a deletion 
"Commit" (see figure 8) at the end of a session. The sec- 
ond option is like the deletion feature of the audkj (tele- 
phone) interface where messages to be deleted can be 
listened to again thereby removing the delete flag and 
only those messages flagged for deletion are deleted 
when the audio session is ended. 

The present inventkjn also allows different types of 
m essages to be bundled together into a multimedia type 
message with multiple body parts. The state information 
for a message includes body part information which in- 
dicates the content type of the body part. 

The present invention is also suitable for providing 
electronic data interchange (EDI) services where EDI 
forms, such as purchase order forms, are provided to a 
user for the purchase of goods, etc. Other types of data 
such as weather data can also be stored and transmit- 
ted. 

The present invention, using the administratton fea- 
tures, can be configured through the network interface 
to perform operations such as sending standard text or 
voice messages to doctor's patients. 

The administration of mailbox features, such as the 
password, telephone ring count, etc., is performed using 
HTML templates that can be customized for each serv- 
ice provider. 

The present invention provides priority of access to 
a mailbox by one of the owners to accesses that are 
made through the telephone interface. 

The browser 144, if it automatically requests a re- 
f r sh of a currently displayed page, allows a page to be 
created that includes a message list icon that can be 
updated to reflect that a new message has arrived dur- 
ing the session. 

The present invention also includes an automatic 
log-off feature that will log a subscriber out of the system 
when there has been no activity for a period of tim . This 
allows subscribers to inadvert ntly I ave their PC 
logged in to the syst m, such as when going horn from 
work, and prev nt others from accessing the syst m 
during the absence. 



The administration features of the system allows a 
V rified system administrator to access a system admin- 
istration home page showing variables sUch as alarms, 
number of users logged in. etc. and to perforrh functions 
such as releasing an access block on a subscriber mail- 
box. 

The present invention has been described with re- 
spect to handling text messages such as e-mail and fac- 
simile. The text messages could have other formats 
such as a Word processing format, a spread sheet for- 
mat, a database format, etc. and could be other MIME 
type informatk^n than can be retrieved without conver- 
sion. The applications that are performed by the appli- 
cation processing units could include personal informa- 
tion managers, appointments, address bocks, etc. 

The many features and advantages of the invention 
are apparent from the detailed specification and, thus, 
it is Intended by the appended claims to cover all such 
features and advantages of the inventicn which fall with- 
in the true spirit and scope of the invention. Further, 
since numerous nrradifrcations and changes will readily 
occur to those skilled in the art, it is not desired to limit 
the invention to the exact construction and operation il- 
lustrated and described, and accordingly all suitable 
modifications and equivalents may be resorted to, falling 
within the scope of the invention. 



Claims 

30 

1. A message storage system, comprising: 

a multimedia mailbox storing voice messages 
and text messages; 

a voice interface providing access to the mes- 
sages via a telephone; and 
a network interface providing access to the 
messages over a network via a persor^al com- 
puter. 

40 

2. A system as recited in claim 1 , wherein said network 
interface comprises an Internet sewer and said 
voice interface comprises a voicemail server. 

45 3. A system as recited in claim 2, wherein said network 
interface provides a home page and an access by 
a user of the home page by an I ntemet browser gets 
an audio voice message when a the user selects a 
voice message and gets an image when the user 

5^ selects a text message. 

4. A system as recited in claim 3. wherein said home 
page is refreshed each time it is accessed. 

55 5. A system as recited in claim 4, wh rein said home 
page includes an expired expiration date. 

5. a system as recited in claim 1, wherein said mailb x 
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stores still and motion vid o nn ssages.- 

7. A systemas recited in claim 4, wherein said network 
interface streams the messages to the user. 

8. A system as recited in claim 7= further comprising a 
personal computer including a prowser which plays/ 
displays messages streamed to the user. 

9. A system as recited in claim 1 , wherein said voice 
Interface outputs the voice and text messages as 
audio to said telephone. 

1 0. A system as recited in claim 1 . wherein the network 
interface includes a data conversion cache for stor- 
ing converted messages. 

11. A system as recited in claim 1 , wherein said network 
interface provides for recording messages. 

1 2. A system as recited in claim 1 . wherein message 
identifiers include a session number and a random- 
ly assigned identifier 

1 3. An apparatus, comprising: 

a message storage system including a multi- 
media mailbox storing voice messages and text 
messages, providing access to the messages via a 
telephone, and providing access to the messages 
via a personal computer 

14. A message storage and retrieval system, compris- 
ing: 

a telephone: 

a telephone network coupled to said telephone; 
a computer including a digital network browser; 
a digital network couplable to the computer; 
and 

a distributed architecture message system cou- 
pled to sad telephone network and said digital 
network, said message system comprising: 
a digital switch coupled to said telephone net- 
work; 

a control unit storing addresses of multimedia 
messages including voice, text, and video mes- 
sages, and controlling switching of said digital 
switch; 

a processing unit coupled to said digital switch, 
storing and retrieving the multimedia messag- 
es, and outputing the voice and text messages 
to said telephone as audio over said telephone 
network responsive to telephone commands; 
a local network coupled to said control unit and 
said processing unit; and 
a network unit coupled to the digital network 
and said local n twork in eluding a data conver- 
sion cache, providing a home page with a mes- 



sage list including messag identifiers compris- 
ing a session number and a randomly assigned 
file identifier responsive to a home page re- 
quest by the browser, retrieving the messages 

5 from the processing unit and streaming the 

messages to the computer responsive to 
browser message play requests to the home 
page having an expired expiration date, the 
computer playing an audio of the voice mes- 

10 sages in real-time as the voice messages are 

received, displaying an image of the text mes- 
sages in real-time as the text messages are re- 
ceived and displaying images of video messag- 
es in real-time as the video messages are re- 

75 ceived, and said computer recording a mes- 

sage and forwarding a message to said net- 
work unit for storage in said processing unit. 

1$. A storage media, comprising: 
^0 a multimedia mailbox storing voice messages 

and text messages, a process providing access to 
the messages via a telephone, and a process pro- 
viding access to the messages via a personal com- 
puter. 

is. A method, comprising: 

providing access to messages in a multimedia 
mailbox storing voice messages and text mes- 
30 sages via a telephone; and 

providing access to the messages via a person- 
al computer 
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f57^ A unified messaging system that provides a 
LLedia mailbox. The system allows a subscriber to 
acc ssscoredmunimediamessages.suchasvo.ceman 
messages, facsimile messages, combined vou:e and 
facsimHe messages and video messages, not only 
throughapublic switched telephonenelworkusingate- 

epho^ebut also over adata network, such as t^m Inter- 
net or an intranet, using a personal computer. The sys- 
tem provides voicemail access over the telephone net- 
work Scafng message number, etc. with the abn.ty to 
play messages to the telephone user as desKed^ For 
text type messages, such as facsimile and e-rnail. the 
systern converts the text into speech and plays the 
sp ech to the telephone user. The system allows a per- 
sonal computer user to obtain the data network access 
using an Internet browser. The browser is used to ac- 
cess a home page of the system and get information 
about the messages stored, and is used to download 
(q I) and play th m ssages at the personal computer 
via data streaming in th cas of a voice or video rn s- 
saqes or vi w the m ssages in the cas of text type 
messages, such as facsimile and e-mail. The us r can 
also perform the other typical messaging functions over 
the data network connection that are provided for tele- 
phone access, such as viewing a message list, saving 
and deleting messages, group list administration and 



other administratton tasks. 
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